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Abstract 

Universiti Kebangsaan Malaysia (UKM) is a research university that continuously undergoes an audit and 
accreditation process for the management of its courses. The Faculty of Engineering and the Built Environment 
(FKAB) is subjected to such processes, one of them is the auditing conducted by the Engineering Accreditation 
Council (EAC), which gives recognition to engineering programmes in Malaysia. The criteria that have been set 
by the EAC requires the faculty to measure and evaluate graduates based directly on the course outcomes (CO) 
and programme outcomes (POs) to ensure that they are able to achieve the programme and course objectives. 
This paper discusses a method developed for the measurement and evaluation of POs which is a direct 
measurement for assessing the courses offered in the mechanical engineering programme. Assessment templates 
are developed using a Microsoft Excel based on the data presented on the grade coordination meetings 
conducted at the end of each semester. The data used for this study were semester 2 2010/2011 session, and 
semester 1 session 2011/2012. Based on the results obtained, the majority of the courses achieves more than 80% 
for course level monitoring for their individual PO achievement except for five subjects. For programme level 
monitoring, from the average POs marks, all POs were scored more than 80%, which indicates that the 
Mechanical Programs offered by the department are satisfied with the department standard. 

Keywords: accreditation, EAC, programme outcome 

1. Introduction 

All programmes in institutes of higher learning and universities are required to implement an outcome-based 
education curriculum set by the Ministry of Fligher Education to ensure quality graduates are produced (Shahrir 
et ah, 2008). The Faculty of Engineering and the Built Environment (FKAB) has initiated the concept of 
Outcome-Based Education (OBE) in 2005, particularly because the FKAB has to ensure that programmes are 
recognized by authorized accreditation bodies. The Engineering Accreditation Council (EAC) is one of 
Malaysia’s accreditation bodies and is responsible for ensuring quality engineering programmes at local 
universities. The EAC has set certain standards to be achieved, emphasizing the measurement of student 
achievement through the OBE and in a manner that is consistent with the requirements of the ministry (Siti 
Aminah et ah, 2011). 

Programme Outcome can be measured by a variety of methods either on questionnaires, grading courses, 
observation of student achievement, portfolio analysis and interviews. All of these methods can be divided into 
two approaches, namely the direct or indirect measurement (Breslow, 2007). Allen (2008) explains that the 
direct measurement methods to assess the students' work in the form of creative assignments and systematically 
can be assessed based on rubric. While the indirect measurement methods can assess students' perception of 
learning and teaching environment. The result of this method is often combined with the direct method results. 
According Wojtczak (2002), Programme Outcome measurement is useful to assess student achievement to meet 
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the needs of external examiners and also can provide the information for the purpose of improving the 
curriculum, as well as to document the success or otherwise for a programme. 

Based on the OBE approach, the measurement of student achievement upon completion of the course is a 
requirement to determine the ability of the students based on their cumulative grade point average measurement 
(CGPA) for each semester. Many of researcher assess the Programme Outcome and Course Outcome based on 
their student achievement (Ishak et al.,2011, Nurina Anuar et al,.2011, Juridah Johari et al, 2010, Hamimi 
Fadziati et al,.2010, Mohd Faizal et al, 2012) . OBE also specifies that the assessment results for each course 
[course outcomes (COs)] will have to depend on what has been stated in the programme handbook. In addition, 
CO measurement is directly dependent on the achievement of students in each assessment determined in each 
course as scores from quizzes, tutorials, laboratories, and examinations. Thus, to ensure that the end result is 
achieved with quality assured, teaching and learning plans should be designed and provided by the course 
instructor at the beginning of each semester (Rozeha et al., 2007). 

Each course offered throughout the mechanical engineering programme has its own COs to be achieved by the 
students upon course completion. Every course is designed to relate the results and contribute to the achievement 
of programme outcomes (POs). A detailed syllabus for each course as well as the respective results of the course 
and its relationship with the PO can be seen in FKAB undergraduate handbook for each academic session. 
Figure 1 shows the process flow of the presentation and evaluation of student performance and achievement of 
COs as practiced in the department. 
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Figure 1. Process of delivery and assessment of student performance and achievement as practiced in the 

department 


Figure 1 shows that every course has specific delivery and evaluation methods for student achievement. Each 
element of assessment must be performed, and the course instructor must ensure the measurement results of the 
course evaluation achieve a predetermined threshold at the departmental level (EAC Document JKMB 2012). 


Table 1. List of POs of the mechanical engineering and manufacturing engineering programmes 

List of Programme Outcomes 

POl: Ability to apply knowledge of mathematics, science, and engineering 

P02: Ability to identify, formulates, solve, and improve engineering problems 

using techniques, skills, and modern engineering tools necessary for 
engineering practice 

P03: Ability to design a component, system, or process to meet desired needs 

P04: Understanding of professional and ethical responsibility of knowledge of 

environment and contemporary issues 
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P05: Ability to understand and apply in-depth knowledge of one or more area of 

specialization in mechanical engineering/manufacturing 

P06: Ability to design and conduct experiments as well as to analyse and interpret 

data 

P07: Ability to communicate and to function effectively in a team 

P08: Recognition of the need for, and ability to tap into lifelong learning 

P09: Knowledgeable in project management, administration, business acumen, and 

entrepreneurship 


In the course assessment, each CO contribution to that particular course is mapped to the relevant PO. This 
enables the evaluation and its contribution toward achieving the PO for the four years of study. Table 1 shows 
the list of POs for the mechanical engineering programme beginning in semester 2010/2011 (FKAB 
Undergraduate Handbook 2011). 

For each programme, such as the mechanical engineering programme for the second semester of 2010/2011, 
approximately 20 courses are offered including optional elective subjects. The general role of the programme 
coordinator is to monitor whether every course offered achieves the COs and PO according to prescribed 
standards. This paper discusses a template developed to help the coordinator monitor the performance of each 
subject offered within a semester. The huge number of courses makes it difficult for the programme coordinator 
to conduct an overall monitoring. However, this template aids in the process of monitoring the expected PO 
performance of each subject and prevents any course from being overlooked. 

2. Methodology 

The measurement procedure was developed using Microsoft Excel format based on the standards established by 
the department. The template contains the course code column representing the entire departmental programme 
as well as the percentage fields pertaining to the achievement of outcomes for each course, as shown in Table 2. 

Coordinator for each course has to determine his/her CO relationship to POs as stated in the Undergraduate 
Handbook. They have to define the percentage workload or assessment for every COs towards the particular POs 
marks. The relationship between POs and COs are static, but the percentage contribution from each COs are 
varies depending on the instructional planning. Figure 2 shows an example of the weightage for each CO to the 
PO for subject KM 1914. The correlation between each PO and CO for each course is outlined in the FKAB 
undergraduate handbook for each semester. For instance, P03 consist of 50% assessment marks from C04 and 
C06 respectively. PO scores were obtained from the scores that have been determined during the grade 
coordination meetings using scoring templates developed by previous researchers (Shahrir, 2007). Every 
student’s assessment marks such as from examination, tutorial or quiz will contribute to their COs marks 
depending on the weight distribution between the assessments. Then, the individual POs marks will generate 
based on the individual COs marks and the weight dissemination between each CO for the particular PO. 


PO 

Hasil Program 

COI 

C02 

C03 

C04 

C05 

C06 

Jumlah 

POI 

Berupaya mengaplikasi pengetahuan tentang 
matematik, sains dan kejuruteraan 

15 

15 

15 

15 

20 

20 

100.00 

P02 

Berupaya mengenalpasti, memformulasi, 
menyelesaikan dan meningkatkan masalah 
kejuruteraan menggunakan teknik-teknik, 
kemahiran-kemahiran, dan perkakasan 
kejuruteraan yang mustahak untuk amalan 
kejuruteraan 



30 

30 


40 

100.00 

P03 

Berupaya untuk merekabentuk satu 
komponen. sistem atau proses untuk 
memenuhi keperluan-keperluan diingini 




50 


50 

100.00 

P04 

Pemahaman mengenaitanggungjawab 
profesional dan beretika daripada ilmu 
pengetahuan tentang alam sekitar dan isu-isu 
kontemporari 







0.00 

P05 

Berupaya untuk memahami dan mengaplikasi 
pengetahuan mendalam samada satu atau 
lebih bidang pengkhususan dalam 
kejuruteraan pembuatan 




50 


50 

100.00 


Figure 2. Sample entry weights that determine the relationship between COs and POs 
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1.6 • 

1.6 ... . 




Figure 3. Final percentages for student achievement of POs for the course KM 1914 


For programme monitoring at the individual level, the department has set that every student should get more than 
50 marks to be considered archiving that particular PO. Next, from the individual PO marks achieved by each 
student, the percentage scoring for the PO measurement of each course measure was calculated based on the 
total percentage of students who achieve at least 50 marks over the total number of students taking the course. 
Figure 3 shows the final marks achieved by the course at the end of the programme. 

The scores for each course will then be transferred to a monitoring template to determine the PO achievement for 
the whole programme. For programme monitoring at the course level,the department has designated that a PO is 
achieved if the percentage of students who scored at least 50 marks in a subject is 80%. Any subject that fails to 
achieve 80% will be marked in yellow, as shown in Table 2, to alert the course lecturer and programme 
coordinator about the PO-related achievement of the course so that further actions could be taken. 

Flowever, to measure the PO for programme monitoring at the programme level, the average of these data, 
excluding the POs for elective subjects, is calculated. In addition, to obtain a more accurate record, the average 
PO for selected subjects is also calculated. 

3. Results and Discussion 

The department offers two programmes: mechanical engineering and manufacturing engineering. To accomplish 
the self-EAC 2012 report, the department requires an approach for monitoring every course for each semester 
and for both programmes for 2010/2011 and for semester 1 2011/2012. However, in this paper, only the 
mechanical engineering programme is discussed with a focus on departmental compulsory courses and faculty, 
excluding compulsory university courses. 


Table 2. Percentage of students who scored a minimum of 50 marks for specific courses for the mechanical 
engineering programme for semester 2 2010/2011 

PO MARKS 


CODE 

POl 

P02 

P03 

P04 

P05 

P06 

P07 

P08 

P09 

KKKM1024 

100 

100 

98.5 

- 

100 

- 

- 

- 

- 

KKKF1121 

- 

- 

- 

100 

- 

- 

- 

- 

- 

KKKM1324 

60.3 

- 

- 

- 

- 

91.2 

- 

- 

- 

KKKM1344 

95.7 

88.4 

- 

- 

- 

- 

- 

- 

- 

KKKJ2324 

70 

90 

- 

- 

70 

- 

98.33 

- 

- 

KKKJ2344 

91.8 

96.7 

- 

- 

98.4 

95.1 

98.4 

- 

- 

KKKJ2724 

88.1 

90.5 

- 

- 

- 

90.5 

90.5 

- 

- 

KKKJ3124 

73.3 

83.3 

75.0 

- 

100.0 

71.7 

95.0 

- 

- 

KKKJ3164 

93.3 

95.0 

- 

- 

88.3 

90.0 

- 

100.0 

- 

KKKJ3344 

100 

100 

- 

- 

100 

100 

- 

- 

- 

KKKJ3944 

96.7 

100.0 

100.0 

- 

75.4 

- 

98.4 

- 

- 
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KKKJ4023 

100.0 

98.3 

98.3 

100.0 

100.0 

100.0 

100.0 

100.0 

- 

KKKJ4703 

96.6 

96.6 

- 

96.6 

96.6 

- 

96.6 

96.6 

96.6 

KKKJ4543 

87.2 

82.1 

- 

- 

82.1 

87.2 

- 

- 

- 

KKKJ4373 

93.3 

100.0 

100.0 

- 

100.0 

- 

- 

- 

- 

KKKJ4363 

86.7 

90.0 

86.7 

- 

- 

86.7 

- 

- 

- 

KKKJ4513 

92 

- 

92 

88 

88 

92 

92 

92 

88 

KKKJ4523 

100 

100 

- 

- 

- 

- 

100 

- 

- 

KKKJ4413 

- 

84.4 

90.6 

- 

84.4 

90.6 

90.6 

- 

- 

KKKJ4163 

95.5 

100 

- 

81.82 

- 

100 

100 

- 

- 


Table 3. Percentage of students who scored a minimum of 50 marks for specific courses for the mechanical 
engineering programme for semester 1 2011/2012 


CODE 

POl 

P02 

P03 

P04 

PO MARKS 

P05 P06 

P07 

P08 

P09 

KKKF1111 

- 

- 

- 

- 

- 

- 

100 

- 

- 

KKKM1513 

60.3 

- 

- 

- 

74.4 

98.7 

98.7 

- 

- 

KKKM1914 

98.4 

98.4 

98.4 

- 

98.4 

- 

- 

- 

- 

KKKM2013 

100 

- 

- 

- 

- 

100 

- 

- 

- 

KKKM2314 

97.1 

- 

- 

- 

95.7 

98.6 

98.6 

- 

- 

KKKM2114 

87.3 

96.8 

- 

- 

81 

93.7 

- 

- 

- 

KKKM2513 

95.6 

100 

- 

- 

- 

82.6 

- 

- 

- 

KKKF3283 

- 

98.8 

- 

98.8 

- 

- 

98.8 

98.8 

- 

KKKJ3143 

95.2 

97.6 

92.9 

- 

97.6 

92.9 

92.9 

- 

- 

KKKJ3333 

100 

100 

- 

- 

100 

100 

100 

- 

- 

KKKL3004 

91.7 

85.0 

- 

- 

96.7 

- 

85.0 

- 

- 

KKKJ3933 

81.8 

100 

100 

- 

81.8 

- 

100 

- 

- 

KKKJ4353 

95.2 

96.8 

- 

- 

96.8 

96.8 

- 

- 

- 

KKKJ4953 

96.6 

100 

100 

- 

- 

- 

100 

100 

100 

KKKJ4013 

100 

100 

100 

100 

100 

100 

100 

100 

- 

KKKJ4573 

83.3 

88.9 

- 

- 

94.4 

- 

- 

- 

- 

KKKJ4553 

77.8 

77.8 

- 

- 

77.8 

- 

- 

- 

- 

KKKJ4364 

100 

93.8 

100 

- 

- 

100 

- 

- 

- 

KKKJ4143 

100 

100 

100 

100 

100 

100 

- 

- 

94.4 


PO assessments are correlated to the CO evaluation based on measurements that have been specified in the 
delivery and evaluation of the course at the beginning of each semester. Lecturers for courses provide a certain 
percentage of each element of evaluation (e.g., assignments, examinations, etc.) that contribute to the COs. 
Percentage weighting may be found in the teaching files for each semester. The percentage of students who 
obtained a minimum score of 50 marks is derived from the grading template and then transferred to the 
monitoring template. 

This type of monitoring can provide an overview of each subject offered to ensure the quality of PO delivery. 
Tables 2 and 3 show the PO performance of individual courses offered by the mechanical engineering 
programme over two consecutive semesters. The listed courses were offered in the specified semesters. For 
almost all courses, 80% of students met the required standards by scoring a minimum of 50 marks for individual 
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PO. However, five subjects, KKKM1324, KKKM2324, KKKJ3124, KKKM1513, and KKKJ4553, failed to 
meet the departmental standards. The small marks on POs, was related to the average score grade achieved by 
the student. If the average marks were low, it also indicates a low POs marks. All the marks were depended with 
the course assessment and low average may because of a lot of factors such as the type of assessment of the 
course, level of question being asked, the attitude of the students, the successfully of course delivery and much 
more. The PO achievement results will be discussed during the meeting of the quality unit (UP3) each semester 
(six-month cycle) after grading is completed for the course. The department, through the UP3 committee, 
identifies the causes of low PO achievement and informs the respective lecturers so that corrective action could 
be taken. Every course subject need to report their POs and COs course achievement in the course file for the 
department and faculty record. They also need to discuss the reason behind their low POs and COs marks (if 
applicable) and the action taken to counter back the deficit or the improvement can be made in the future. 
Changes and improvements in the delivery and assessment methods for the course are undertaken as part of a 
continuous quality improvement process for the programme in JKMB. 

For programme level monitoring, a measurement of each PO for every course can be conducted. However, 
selecting certain core courses that represent the measurement of a certain PO is more appropriate. The choice of 
courses is determined by the department based on the courses’ importance in terms of contribution to a particular 
PO. For example, to represent POl, the courses considered are the KM1344, KM2114, KJ3124, KJ3933, and 
KJ4953, whereas for P09, KJ4703 and KJ4953 are considered. For each PO, at least two courses are selected to 
provide a cumulative measurement. 

Figure 4 shows the average percentage of student achievement for each PO based on a particular subject or all 
subjects, excluding elective courses. PO is measured based on the CO’s the relationship with and contribution to 
the PO (CO attainment directly measured for each course). From figure 4, it shows that all POs exceeded the 
target of 80%. POl recorded the lowest achievement at 86.9% for selected subjects and 90.1% for all all 
subjects. 


100 

90 

80 

70 

• 60 
1 

50 

40 

30 

20 

10 

0 


8 


PO percentage achievement 


*1 


on cn 


I 4 


2 58 
s -g 


Programme Outcome 
3 Selected Subject ■ All Subjects 


Figure 4. Percentage of student achievement for each PO (Achievement for 50 marks and above) 


There are quite difficult to indicate student PO achievement beside the direct measurement from academic 
assessment and indirect measurement which is student perception. However, high student PO achievement can 
be demonstrated indirectly through the active participation of students in numerous national and international 
competitions such as the Perodua Eco-Challenge (2009, 2011, and 2012), Shell Eco-Marathon Asia (2010, 2011, 
and 2012), Proton Green Mobility Challenge 2012, and Malaysian Autodesk Design Competition (2010 and 
2011 ). 


4. Conclusion 

All POs have generally scored over 80% which are satisfied for programme level monitoring. The results are 
based on the theoretical aspect (class lectures) and the practical aspect (exposure through participation in 
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competitions and association. However, for course level monitoring, a number of courses can further enhance 
their PO achievement. This comprehensive monitoring system ensures that each course’s PO achievement is 
determined by the department. The department can identify the elements of underachievement and then inform 
concerned lecturers to take action for the purpose of continuous quality improvement, as intended for all EAC 
report documentations. 
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